Image display apparatus, image-taking apparatus and image display method

ABSTRACT

The image display apparatus according to an aspect of the present invention comprises: an image input device which inputs an image signal; a particular target detection device which detects a particular target included in the image signal based on a particular target evaluation value indicating the feature of the particular target; a frame display information generation device which generates frame display information indicating a frame surrounding the detected particular target and which causes the frame to change continuously or by stages according to the particular target evaluation value; and a display device which displays the frame based on the generated frame display information. That is, by causing the frame to change continuously or by stages according to the evaluation value of a particular target, it is possible to avoid sudden change in the frame display.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. Ser. No. 17/067,292 filed onOct. 9, 2020, which is a Continuation of U.S. Ser. No. 16/701,958 filedDec. 3, 2019 (now U.S. Pat. No. 10,841,499 issued Nov. 17, 2020), whichis a Continuation of U.S. application Ser. No. 15/642,646 filed Jul. 6,2017 (now U.S. Pat. No. 10,554,894 issued Feb. 4, 2020), which is aDivisional of U.S. application Ser. No. 14/198,358 filed Mar. 5, 2014(now U.S. Pat. No. 9,736,379 issued on Aug. 15, 2017), which is aDivisional of U.S. application Ser. No. 11/730,140 filed on Mar. 29,2007 (now U.S. Pat. No. 8,704,856 issued on Apr. 22, 2014), which claimspriority under 35 U.S.C. § 119(a) to Application No. 2006-095816 filedin Japan on Mar. 30, 2006, all of which are hereby expresslyincorporated by reference into the present application.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an image display apparatus, animage-taking apparatus and an image display method, and in particular toa technique for displaying information indicating a particular areaincluded in an image.

Description of the Related Art

Here, an image-taking apparatus (a digital camera) will be described asa conventional art for a display apparatus and an image-takingapparatus. Digital cameras which can detect a particular area (a face)from a subject image have been put to practical use. Such digitalcameras can detect a face and perform automatic focus control (AF) andautomatic exposure control (AE) for the face area to obtain a subjectimage with an appropriate focus and exposure even when it is difficultto perform AF and AE by normal image-taking. Furthermore, when a face isdetected, a frame is displayed for the detected area, so that anoperator can determine whether or not the digital camera has detected aface.

In Japanese Patent Application Laid-Open No. 2005-286940, a digitalcamera is disclosed which makes it possible to detect a face areaexisting in a subject image obtained by image taking and let the userknow the face area when he performs image taking. AF and AE areperformed with the use of the image data indicating the image within aframe. It is possible to record image data indicating a subject image inwhich the face area is appropriately focused and which has appropriatebrightness.

In Japanese Patent Application Laid-Open No. 2005-284203, a digitalcamera is disclosed which detects a face area existing in a subjectimage obtained by image taking, and which is provided with a lenscontrol device that controls a lens driving circuit to position a lensat a position where a face evaluation value becomes the largest.

SUMMARY OF THE INVENTION

In these digital cameras, an operator can determine whether or not thedigital camera has detected a face from the display of the frame.However, in the case where the detection situation suddenly changes, theframe may be suddenly displayed or hidden. For example, in the case oftaking a portrait as shown in FIGS. 34A and 34B, a face is detected anda frame is displayed when hair flowing in the wind is not over an eye asin FIG. 34A. However, when hair flowing in the wind is over an eye as inFIG. 34B, the face is not detected because the one eye is not detected,and the frame is not displayed. If this situation is repeated, the frameblinks. Similarly, in the case of taking an image of a child who doesnot keep still, the frame blinks annoyingly.

In the technique described in Japanese Patent Application Laid-Open No.2005-284203, the lens is moved so that the face evaluation value becomesthe largest so that the face of a person is focused. It is not atechnique for letting an operator know the face area when he performsimage taking.

The present invention has been made in consideration of the abovesituation, and its object is to provide an image display apparatus, animage-taking apparatus, and an image display method which are capable ofimproving the conventional frame display in which only distinctionbetween display and non-display is made and preventing annoying displayas far as possible, by causing a frame surrounding a particular targetto change continuously or by stages according to particular targetevaluation value.

The image display apparatus according to a first aspect of the presentinvention comprises: an image input device which inputs an image signal;a particular target detection device which detects a particular targetincluded in the image signal based on a particular target evaluationvalue indicating a feature of the particular target; a frame displayinformation generation device which generates a frame displayinformation indicating a frame surrounding the detected particulartarget and which causes the frame to change continuously or by stagesaccording to the particular target evaluation value; and a displaydevice which displays the frame based on the generated frame displayinformation.

That is, by causing the frame to change continuously or by stagesaccording to the evaluation value of a particular target, it is possibleto avoid sudden change in the frame display.

The image display apparatus according to a second aspect of the presentinvention is the image display apparatus according to the first aspect,wherein the frame display information generation device generates theframe display information which causes at least any of a shade intensityof a frame line, a hue of a frame line, a line segment display rate ofline segments constituting a frame line and a frame line width, of theframe to change according to the particular target evaluation value.

That is, due to the graphical change in the frame display, such aschange in the shade intensity of the frame line, change in the hue ofthe frame line, change in the line segment display rate of the linesegments constituting the frame line and change in the frame line width,it is possible to avoid sudden change in the frame display. The linesegment display rate of the frame line refers to, when the frame line isindicated by a dashed line (a dotted line), the ratio of the length ofone dash (dot) to the interval between its one end and the same-side endof an adjacent dash (dot). If the entire frame line is indicated by asolid line, the line segment display rate is 100%.

The image display apparatus according to a third aspect of the presentinvention is the image display apparatus according to the first aspect,wherein the frame display information generation device generates theframe display information which causes a temporal display rate of aframe line of the frame to change according to the particular targetevaluation value.

That is, by causing the temporal display rate of the frame (the ratio ofthe frame line display time to a unit time) to change according to theparticular target evaluation value, it is possible to avoid suddenchange in the frame display.

The image display apparatus according to a fourth aspect of the presentinvention is the image display apparatus according to the first orsecond aspect, wherein an indicator which shows correspondence between adisplay of the frame and the particular target evaluation value isfurther displayed on the display device.

That is, the particular target evaluation value can be known from thedisplay of the frame.

The image display apparatus according to a fifth aspect of the presentinvention is the image display apparatus according to any of the firstto fourth aspects, wherein a display which indicates the particulartarget evaluation value is further shown on the display device.

That is, the particular target evaluation value can be known morespecifically.

The image display apparatus according to a sixth aspect of the presentinvention is the image display apparatus according to any of the firstto fifth aspects, wherein when multiple particular targets are includedin the image signal, the frame display information generation devicegenerates the frame display information which indicates each of framessurrounding the particular targets with use of each of at least twodifferent colors.

That is, when there are multiple particular targets, the particulartarget evaluation value of each particular target can be identified.

The image display apparatus according to a seventh aspect of the presentinvention comprises: an image input device which inputs an image signal;a particular target detection device which detects a particular targetincluded in the image signal based on a particular target evaluationvalue thereof; a frame display information generation device whichgenerates a frame display information indicating a frame surrounding thedetected particular target; and a display device which synthesizes anddisplays, on the image, the frame based on the generated frame displayinformation together with a numerical value indicating the particulartarget evaluation value.

That is, the particular target evaluation value can be directly known asa numeral.

The image display apparatus according to an eighth aspect of the presentinvention is the image display apparatus according to any of the firstto seventh aspects, further comprising a display determination devicewhich determines whether or not to display the frame based on theparticular target evaluation value, wherein the frame is displayed whenthe particular target evaluation value is equal to or above apredetermined display determination value.

That is, by not displaying the frame when the evaluation value of theparticular target is low, only necessary frame display can be shown.

The image display apparatus according to a ninth aspect of the presentinvention comprises: an image input device which inputs an image signal;a particular target detection device which detects a particular targetincluded in the image signal based on a particular target evaluationvalue indicating a feature of the particular target; a frame displayinformation generation device which generates a frame displayinformation indicating a frame surrounding the detected particulartarget; a display device which displays the frame based on the generatedframe display information; and a display determination device whichdisplays the frame when the particular target evaluation value is equalto or above a predetermined display determination value; and the framedisplay information generation device generates the frame displayinformation so that the frame gradually disappears with a lapse of timewhen it is determined by the display determination device to switch fromdisplay to non-display of the frame, and the frame gradually appearswith a lapse of time when it is determined to switch from non-display todisplay of the frame.

That is, if it is determined by the display determination device toswitch from display to non-display of the frame indicating theparticular target, the frame surrounding the detected particular targetis displayed in a manner that it gradually disappears with a lapse oftime, and thereby sudden change in the frame display can be avoided.Furthermore, if it is determined by the display determination device toswitch from non-display to display of the frame indicating theparticular target, the frame is displayed in a manner that it graduallyappears with a lapse of time, and thereby sudden change in the framedisplay can be avoided.

The image display apparatus according to a tenth aspect of the presentinvention is the image display apparatus according to the ninth aspect,wherein the frame display information generation device generates theframe display information which causes at least any of a shade intensityof a frame line, a line segment display rate of line segmentsconstituting a frame line, a frame line width, a display magnificationrate of the frame and a temporal display rate of a frame line to changeaccording to a lapse of time from a point of time when a determinationto switch from display to non-display is made or a point of time when adetermination to switch from non-display to display is made.

That is, by the frame display information generation device causing atleast any of the shade intensity of the frame line, the line segmentdisplay rate of the line segments constituting the frame line, the frameline width, the display magnification rate of the frame and the temporaldisplay rate of the frame line to change according to the lapse of timefrom the point of time when the determination to switch from display tonon-display is made or the point of time when the determination toswitch from non-display to display is made, sudden change in the framedisplay can be avoided.

The image display apparatus according to an eleventh aspect of thepresent invention is the image display apparatus according to the ninthaspect, wherein the frame display information generation devicegenerates the frame display information which causes an in-screendisplay rate of the multiple line segments constituting a frame line ofthe frame to change according to the lapse of time from the point oftime when the determination to switch from display to non-display ismade or the point of time when the determination to switch fromnon-display to display is made.

That is, by the frame display information generation device causing thein-screen display rate of the multiple line segments constituting theframe line of the frame to change according to the lapse of time fromthe point of time when the determination to switch from display tonon-display is made or the point of time when the determination toswitch from non-display to display is made, sudden change in the framedisplay can be avoided.

The image display apparatus according to a twelfth aspect of the presentinvention is the image display apparatus according to the eleventhaspect, wherein the frame display information generation devicegenerates the frame display information which causes the distance ofeach of the multiple line segments constituting the frame line of theframe from the center of the frame to change according to the lapse oftime.

That is, by the frame display information generation device causing thein-screen display rate of the multiple line segments constituting theframe line of the frame to change according to the lapse of time fromthe point of time when the determination to switch from display tonon-display is made or the point of time when the determination toswitch from non-display to display is made, sudden change in the framedisplay can be avoided. It is assumed that the in-screen display rate ofthe frame is 0%, for example, when the frame is displayed in a mannerthat multiple line segments constituting the frame line are separatedand spread almost radially, with a particular target as the center, andgradually moved beyond the screen or when the frame is displayed in amanner that the line segments are moved toward the center of aparticular target to gradually come together.

The image display apparatus according to a thirteenth aspect of thepresent invention is the image display apparatus according to any of theeighth to twelfth aspects, wherein the apparatus accepts an input of thedisplay determination value by an operator.

That is, by an operator inputting the evaluation value of the particulartarget for which a frame is to be displayed, only necessary framedisplay can be shown.

The image display apparatus according to a fourteenth aspect of thepresent invention is the image display apparatus according to any of thefirst to thirteenth aspects, wherein the display device displays animage based on the image signal, and synthesizes and displays the frameon the image.

That is, by synthesizing and displaying an image and a frame, a targetdetected by the particular target detection device can be easilyconfirmed.

The image display apparatus according to a fifteenth aspect of thepresent invention is the image display apparatus according to any of thefirst to fourteenth aspects, wherein the particular target is a face,and the particular target detection device is a face detection device.

That is, a display apparatus capable of avoiding sudden change in thedisplay of the frame indicating a face can be realized.

The image display apparatus according to a sixteenth aspect of thepresent invention comprises: the image display apparatus according toany of the first or fifteenth aspects; and an image-taking device whichfunctions as the image input device.

That is, an image-taking apparatus capable of avoiding sudden change inthe display of the frame indicating the particular target can berealized.

The image display method according to a seventeenth aspect of thepresent invention comprises: obtaining an image signal indicating asubject image; detecting a particular target included in the imagesignal based on a particular target evaluation value indicating afeature of the particular target; generating a frame display informationindicating a frame surrounding the detected particular target; anddisplaying the frame display information indicating the frame which iscaused to change continuously or by stages according to the particulartarget evaluation value, on a display device.

The image display method according to an eighteenth aspect of thepresent invention comprises: obtaining an image signal indicating asubject; detecting a particular target included in the image signalbased on a particular target evaluation value indicating a feature ofthe particular target; generating a frame display information indicatinga frame surrounding the detected particular target; determining whetheror not to display the frame according to the particular targetevaluation value; and generating the frame display information so thatthe frame gradually disappears with a lapse of time when it isdetermined to switch from display to non-display of the frame, and theframe gradually appears with a lapse of time when it is determined toswitch from non-display to display of the frame, and displaying theframe based on the frame display information.

The image display method according to a nineteenth aspect of the presentinvention is the image display method according to the seventeenth oreighteenth aspect, wherein the frame is synthesized and displayed on theimage based on the image signal and the frame display information.

According to the present invention, it is possible to provide an imagedisplay apparatus, an image-taking apparatus, and an image displaymethod which are capable of improving the conventional frame display inwhich only distinction between display and non-display is made andpreventing annoying display as far as possible, by causing a framesurrounding a particular target to change continuously or bay stagesaccording to particular target evaluation value.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an example of the internalconfiguration of an image display apparatus according to the presentinvention;

FIGS. 2A and 2B are block diagrams showing a detailed embodiment of aface detection section in FIG. 1 ;

FIG. 3 is a table showing a standardized face evaluation value andposition information about a frame indicating a face which is obtainedfrom the detection position and the parameter detection positions, thestandardized face evaluation value and the position information beingoutputted from an output device;

FIGS. 4A and 4B show an example of a CPU generating a frame displayindicating a face, from the standardized face evaluation value and theposition of the frame indicating a face, outputted from the facedetection section, and displaying it on a display section;

FIG. 5 shows an example of a table for standardizing the face evaluationvalue;

FIG. 6 is a flowchart showing a procedure for the CPU to display theframe continuously or by stages according to the face evaluation value;

FIG. 7 shows an example where the frame is displayed continuously or bystages according to the face evaluation value, and an indicator is alsodisplayed;

FIG. 8 is a flowchart showing a procedure for the CPU to display theframe continuously or by stages according to the face evaluation valueand display an indicator;

FIG. 9 shows an example where, when the frame is displayed continuouslyor by stages according to the face evaluation value and an indicator isalso displayed, a display indicating the face evaluation value is shownon the indicator;

FIG. 10 is a flowchart showing a procedure for the CPU to, whendisplaying the frame continuously or by stages according to the faceevaluation value and displaying an indicator, show a display indicatingthe face evaluation value on the indicator;

FIGS. 11A and 11B show an example where, when the frame is displayedcontinuously or by stages according to the face evaluation value and anindicator is also displayed, a display indicating the face evaluationvalue is shown on the indicator, and a different display color is usedfor each detection result;

FIGS. 12A, 12B and 12C show an example where, when the frame isdisplayed continuously or by stages according to the face evaluationvalue, the hue of the frame is changed according to the face evaluationvalue;

FIG. 13 is a flowchart of the CPU displaying the frame continuously orby stages according to the face evaluation value, with the hue of theframe changed according to the face evaluation value;

FIG. 14 shows an example where, when the frame is displayed continuouslyor by stages according to the face evaluation value, the line segmentdisplay rate of the line segments constituting the frame line of theframe is changed according to the face evaluation value;

FIG. 15 is a flowchart of the CPU displaying the frame continuously orby stages according to the face evaluation value, with the line segmentdisplay rate of the frame changed according to the face evaluationvalue;

FIG. 16 shows an example where, when the frame is displayed continuouslyor by stages according to the face evaluation value, the frame linewidth of the frame is changed according to the face evaluation value;

FIG. 17 is a flowchart of the CPU displaying the frame continuously orby stages according to the face evaluation value, with the frame linewidth of the frame changed according to the face evaluation value;

FIGS. 18A, 18B and 18C show an example where, when the frame isdisplayed continuously or by stages according to the face evaluationvalue, the temporal display rate of the frame line is changed accordingto the face evaluation value;

FIG. 19 is a flowchart of the CPU displaying the frame continuously orby stages according to the face evaluation value, with the temporaldisplay rate of the frame line changed according to the face evaluationvalue;

FIG. 20 shows an example where, when the frame is displayed continuouslyor by stages according to the face evaluation value, a standardized faceevaluation value is displayed as a numerical value together with theframe line;

FIG. 21 is a flowchart of the CPU displaying the frame continuously orby stages according to the face evaluation value, with a standardizedface evaluation value displayed as a numerical value together with theframe line;

FIG. 22 shows an example where, when the face evaluation value becomesequal to or above the display determination value of the frameindicating a face or below the value, the frame is changed with thelapse of time;

FIG. 23 is a flowchart of the CPU displaying the frame in a manner thatthe frame changes with the lapse of time when the face evaluation valuebecomes equal to or above the display determination value of the frameindicating a face or below the value;

FIG. 24 shows an example where, when the face evaluation value becomesequal to or above the display determination value of the frameindicating a face or below the value, the line segment display rate ofthe frame is changed with the lapse of time;

FIG. 25 is a flowchart of the CPU causing the line segment display rateof the frame to change with the lapse of time when the face evaluationvalue becomes equal to or above the display determination value of theframe indicating a face or below the value;

FIG. 26 shows an example where, when the face evaluation value becomesequal to or above the display determination value of the frameindicating a face or below the value, the frame line width of the frameis changed with the lapse of time;

FIG. 27 is a flowchart of the CPU causing the frame line width of theframe to change with the lapse of time when the face evaluation valuebecomes equal to or above the display determination value of the frameindicating a face or below the value;

FIG. 28 shows an example where, when the face evaluation value becomesequal to or above the display determination value of the frameindicating a face or below the value, the display time rate of the frameis changed with the lapse of time;

FIG. 29 is a flowchart of the CPU causing the display time rate of theframe to change with the lapse of time when the face evaluation valuebecomes equal to or above the display determination value of the frameindicating a face or below the value;

FIG. 30 shows an example where, when the face evaluation value becomesequal to or above the display determination value of the frameindicating a face or below the value, the in-screen display rate of theframe is changed with the lapse of time;

FIG. 31 is a flowchart of the CPU causing the in-screen display rate ofthe frame to change with the lapse of time when the face evaluationvalue becomes equal to or above the display determination value of theframe indicating a face or below the value;

FIG. 32 shows an example where, when the face evaluation value becomesequal to or above the display determination value of the frameindicating a face or below the value, the frame is changed with thelapse of time with the use of multiple means;

FIGS. 33A, 33B, 33C, 33D, 33E, 33F, 33G and 33H show an example of theframe indicating a face and the change in the display of the frame; and

FIGS. 34A and 34B illustrate the conventional display of the frameindicating a face.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

An embodiment of an image display apparatus, an image-taking apparatusand an image display method according to the present invention will bedescribed with reference to accompanying drawings.

FIG. 1 is a block diagram showing an example of the internalconfiguration of an image-taking apparatus 1 as an embodiment of animage display apparatus according to the present invention.

In this figure, an image-taking apparatus 1 is configured by an imageinput section 10, an input signal processing section 12, a CPU 20, adisplay section 22, an image recording section 24, an operation section26, a communication section 28 and a particular target detection section(face detection section) 30. The CPU 20 controls each of the blocks inresponse to an operator's instruction from the operation section 26 orthe communication section 28.

When the image input section 10 is configured as an image-takingsection, the image input section 10 coverts an optical image coming fromthe outside to an image signal with the use of a lens, an image-takingelement and the like and outputs it. A device for a flash orillumination to be used as an auxiliary light in the case of a lowilluminance may be provided. When the image input section 10 isconfigured as an external signal input section for inputting an imagesignal from the outside, the configuration of the image input section 10may be overlapped with the configuration of the image recording section24 or the communication section 28. That is, an image is inputted withthe use of the recording medium interface of the image recording section24 or the communication section 28. In addition, an auxiliary lightsection for emitting an auxiliary light as necessary, a speaker sectionfor reproducing voice or giving an alarm sound and the like are alsoprovided though they are not shown.

The input signal processing section 12 processes the image signal andhands it to the CPU 20. The signal processing performed here differsdepending on the output image signal form required by an operator.Offset correction processing, white balance processing, color matrixprocessing, tone conversion processing, demosaicking processing, outlineenhancement processing, compression processing and the like areperformed for the image signal according to image-taking modes. Theimage-taking modes include a still-image taking mode, a motion-picturetaking mode and a through-image display mode which is the stage ofpreparation for these image-taking modes. Furthermore, the still-imagetaking mode is further separated in a compression recording mode and anon-compression recording mode. The non-compression recording mode isfurther separated in a RAW image recording mode and a non-compressionimage format recording mode. The through image display mode, which is tobe described in detail later, causes the image input section 10 tocontinuously output images, causes the input signal processing section12 to perform signal processing of the outputted continuous images andoutputs the images in an output format corresponding to the display formof the display section 22. Thereby, through images are continuouslydisplayed on the display section 22.

The image recording section 24 records an inputted image signal. As amedium for recording, either an internal recording medium or an externalrecording medium like a memory card may be used. Both of them may exist.Not only an image but also inputted or generated information (data) canbe recorded in the recording medium. In spite of the name of therecording section, it can perform not only storage but also reading.Data read therefrom can be transmitted from the communication section 28or displayed on the display section 22, via the CPU 20.

The operation section 26 is configured by a power source switch andoperation switches for operating the image-taking apparatus 1. Anoperator can operate the image-taking apparatus 1 by giving aninstruction to the image-taking apparatus 1 from the operation section26.

The communication section 28 is a device for the image-taking apparatus1 to communicate with the outside. As a communication method, wiredcommunication, wireless communication, optical communication or the likecan be used depending on the situation. A communication protocol can beselected depending on the situation. The communication section 28 cannot only transmit information to the outside but also receiveinformation from the outside.

The particular target detection section (face detection section) 30detects particular targets of a subject, that is, the face, eyes,pupils, nose, mouth and the like from a through image or a recordedimage and identifies their feature points and patterns to detect theface of the subject. As a specific face detection method, a well-knownmethod can be used, such as a method for detecting a face by detectionof an edge or detection of a form pattern, a feature point vectorapproximation method in which the feature points of each particulartarget described above are vectorized to detect approximation of thefeature point vectors, and a method for detecting a face by detection ofthe hue or detection of the flesh color. When detecting the face of aperson, the face detection section 30 outputs information indicating thedisplay area of the face to the CPU 20 together with a face evaluationvalue, and it can display a frame indicating the display area andaccompanying information on the display section 22 as necessary.

Here, the details of the face detection section 30 will be describedwith the use of the feature point vector approximation method describedabove as an example. FIGS. 2A and 2B show the example of the featurepoint vector approximation method.

FIG. 2A is a block diagram showing an embodiment of the face detectionsection 30 capable of detecting a face and outputting the result aboutwhether a face has been detected or not and the display position andsize of a frame. While moving the detection position, each of parameterdetection processing devices 41 and 42 detects a different parameterfrom an input image signal. A vector synthesis processing device 48synthesizes the detected parameters to generate a vector. For example,if eight kinds of parameters are detected, an eight-dimensional vectoris synthesized. A determination processing device 50 compares thesynthesized vector and a reference vector stored in a dictionary 46,and, if the difference is equal to or below a predetermined value, itdetermines that a face has been detected. It is desirable that multiplereference vectors are stored according to face features. After thedetermination, the result about whether or not it has been determinedthat a face has been detected and the position of a frame indicating theface obtained from the detection position and the parameter detectionpositions are outputted from an output device 52.

FIG. 2B is a block diagram showing an embodiment of the face detectionsection 30 capable of detecting a face and outputting a standardizedface evaluation value of a detected face and the display position andsize of a frame. Since the sections responsible for steps to the step ofgenerating a vector are the same as those in FIG. 2A, descriptionthereof will be omitted. A determination processing device 51 comparesthe synthesized vector and a reference vector stored in the dictionary46, and determines the difference between the vectors to be a faceevaluation value, as an absolute value. If there are multiple referencevectors, a value obtained by further performing synthesis of a vector(sum of squares) of the differences between the respective referencevectors and the synthesized vector is set as the face evaluation value.That is, if the synthesized vector ideally corresponds to the referencevector, 0 is outputted. After the calculation of the face evaluationvalue, the face evaluation value is sent to a standardization processingdevice 54, and the standardized face evaluation value and the positionof the frame indicating a face obtained from the detection position andthe parameter detection positions are outputted from an output device53. It is desirable to appropriately select a method for thestandardization according to the method for displaying a frame to bedisplayed which is to be described alter. In this embodiment,description will be made on the assumption that the face detectionsection shown in FIG. 2B is used. In FIGS. 2A and 2B, the respectiveprocessing sections are separated. However, it is also possible that theCPU 20 performs all the processing of the face detection section. Inthis case, everything is to be processed by software.

FIG. 3 shows the standardized face evaluation value and positioninformation about the frame indicating a face obtained from thedetection position and the parameter detection positions, outputted froman output device 53. Here, the frame position information is assumed tobe a rectangle frame, and the coordinates of the four corners areoutputted. However, it is also possible to output the coordinates of anycorner of the frame, the frame size, the inclination and the like. Aframe in a shape other than a rectangle is also possible. The outputmethod in such a case will be appropriately selected. FIG. 3 showsdetection results 1, 2 and 3, which indicates that the faces of threepeople have been detected.

Next, detailed description will be made on the operation of displayingthe frame indicating a face on the display section 22 of theimage-taking apparatus 1.

First Embodiment of Frame Display

FIGS. 4A and 4B show an example of the CPU 20 generating a frame displayindicating a face, from the standardized face evaluation value and theposition of a frame indicating a face, outputted from the face detectionsection 30, and displaying it on the display section 22. Here, the faceevaluation value is indicated by the shade intensity of the frame line.That is, when the face evaluation value is small, the frame line isdisplayed (A1) with a low shade intensity as shown in FIG. 4A (displayedin white in the figure), and when the face evaluation value is large,the frame line is displayed (A2) with a high shade intensity as shown inFIG. 4B (displayed in black in the figure). When a frame line isdisplayed in grayscale, the shade intensity of the frame linecorresponds to the face evaluation value, and when the frame line isdisplayed in color, the shade intensity of at least any of the colorcomponents constituting the frame line may correspond to the faceevaluation value.

When the display section 22 is capable of displaying, for example, 8-bitimage data, it is recommended to use standardization 1 in thestandardization table in FIG. 5 to display the face evaluation value andthe standardized face evaluation value as described above. That is, whenthe face evaluation value is 0, the output of the standardization 1 is255. If the frame line is displayed in grayscale, then the frame line isdisplayed by 255, that is, in white. If the frame line is displayed ingreen, then the frame line is displayed in deep green because green isindicated by 255 and other colors are indicated by 0. When the faceevaluation value is 100, the output of the standardization 1 is 0. Ifthe frame line is displayed in grayscale, then the frame line isdisplayed by 0, that is, in black. If the frame line is displayed ingreen, the frame line is also displayed in black because both of greenand other colors are indicated by 0. The evaluation value is between 0and 100 here. However, this value differs according to the configurationof the face detection section. Similarly, the standardization table alsodiffers according to the data input specifications required by thedisplay section. That is, display data 255 does not necessarilycorrespond to white. Therefore, it is necessary to define thestandardization table based on the specifications of the display section22 or the specifications required for display. Furthermore, though anevaluation value table with one hundred stages is defined here, a tablewith more detailed stages and a rougher table, for example, with aboutten stages are also possible. Furthermore, in the example in FIG. 5 ,all the face evaluation values equal to or above 100 may be treated as100.

By displaying the frame continuously or by stages according to the faceevaluation value in this way, it is possible to smoothly display theframe, avoiding the conventional sudden display change between displayand non-display of the frame.

Next, description will be made on the content of processing to beperformed in the case of changing the frame indicating a facecontinuously or by stages according to the face evaluation value, by theimage-taking apparatus 1 configured as described above.

FIG. 6 is a flowchart showing a procedure for the CPU 20 to display theframe continuously or by stages according to the face evaluation value.

Signal processing is performed for an image signal obtained by inputtingan image from or taking an image with the image input section 10, at aninput signal processing section to acquire the image signal as an image(step S100). Next, at the face detection section, particular targets ofa subject included in the inputted image, that is, a face, eyes, pupils,a nose, a mouth and the like are detected from the inputted image, andthe feature points and patterns of them are identified to detect theface of the subject (step S102). Next, the image inputted from the imageinput section 10 is displayed (step S104). Next, at step S110, if “thenumber N of detected faces=0” is not satisfied, then the flow proceedsto step S120. “If the number N of detected faces=0” is satisfied, thenthe flow proceeds to step S200.

At step S120, the CPU 20 first substitutes the standardized faceevaluation value of a detection result 1 for the frame line shadeintensity. Then, the flow proceeds to step S186, the frame for thedetection result 1 is displayed on the display section 22 with a frameline for which the shade intensity has been set at step S120. Then, thedetection result 1 is deleted at step S190, and 1 is subtracted from thenumbers of a detection result 2 and subsequent detection results. In theexample of the table in FIG. 3 of standardized face evaluation valuesand position information about the frame indicating a face obtained fromthe detection position and the parameter detection positions, thedetection result 2 is moved up to the detection result 1, the detectionresult 3 is moved up to the detection result 2, and the detection result1 is deleted. Then, the flow proceeds to step S190, and 1 is subtractedfrom the number of detected faces N. Then, the flow returns to step S110and repeats the similar processing until the number of detected faces Nbecomes 0. As a result, the frame line can be displayed with a frameline shade intensity according to the face detection result, and anoperator can know the detection result. Furthermore, it is possible toavoid sudden display change being made depending on whether a face hasbeen detected or not, and natural display can be possible.

At step S200, it is determined whether or not to perform photographing.Usually, the determination is made based on whether a shutter button ora recording button (neither of them are shown) provided on the operationsection 26 has been pushed or not. If photographing processing is to beperformed, then the photographing processing is performed at step S202,and the flow proceeds to step S210. If the photographing processing isnot to be performed, then the flow proceeds directly to step S210, andit is determined whether or not to end the processing. If the processingis not to be ended, the flow returns to step S100, and the similarprocessing is repeated.

Here, acquisition of an image (step S100), detection of a face (stepS102) and display of the image (step S104) are performed in timesequence for convenience of description with reference to the flowchart.However, it is also possible to perform the display of the image (stepS104) while performing the detection of a face (step S102), for example,in the case of such a configuration that the face detection sectionoperates independently from the CPU 20, and it is rather desirable inorder to shorten the entire processing. Not only for the parts describedabove but also for the parts described below, higher speed can berealized by performing parallel processing if possible.

Here, an image is displayed on the display section 22 first, and then aframe is displayed on the display section 22. However, it is alsopossible to first synthesize an image and a frame to be displayed andthen send the synthesized result to the display section 22 to display itthereon, depending on the configuration of the apparatus.

Furthermore, it is not necessarily required to display an image. Forexample, in the case of an image-taking apparatus for monitoringintended to monitor whether the face of a person exists or not, it issufficient to display only a face frame, without displaying an image.Because it is sufficient to display the image only when it is determinednecessary to do so from the displayed frame. For example, the followingis conceivable. When a face is detected, a frame is displayed. Then, ifthe detection continues for a predetermined period of time (for example,three seconds or more), then an image is displayed for the first time.This makes it possible to monitor not a monitoring picture screen onwhich a lot of targets are displayed and which requires alertness butonly a necessary monitoring screen.

First Variation Example of the First Embodiment of Frame Display

Next, FIG. 7 shows an example of the CPU 20 generating, from thestandardized face evaluation value (for example, a face evaluation valuestandardized with the use of standardization 2 in FIG. 5 ) and theposition of a frame indicating a face, outputted from the face detectionsection 30, a frame display indicating a face, displaying the frame onthe display section 22 and displaying an indicator and the likeindicating the correspondence between the frame display and thestandardized face evaluation value.

In FIG. 7 , as examples of the indicator and the like, an example ofdisplaying an indicator I1 in gradation at the right end and showing adisplay of 50% (L1) at the lower right and a display of 100% (L2) at theupper right. An operator (or a user) can know an approximate value ofthe standardized face evaluation value by comparing the displayed frameindicating a face with the indicator and the like. That is, anapproximate value of the face evaluation value can be known based onwhich shade intensity at which position of the indicator I1 displayed ingradation the shade intensity of the frame line indicating a facecorresponds to.

In this embodiment, the frame indicating a face is displayed when thestandardized face evaluation value is equal to or above a displaydetermination value (in the example in FIG. 7 , 50% or above). Thereby,it is possible to display only a frame indicating such a face that theface evaluation value is equal to or above a set display determinationvalue, and avoid complicatedness of display. Furthermore, this displaydetermination value may be selected from among values inputted inadvance or may be inputted by the operator. It is possible to display aface with a lower face evaluation value or only a face with a high faceevaluation value as the operator likes.

FIG. 8 is a flowchart showing a procedure for the CPU 20 to display theframe continuously or by stages according to the face evaluation valueand display an indicator.

Since this embodiment is a variation example of FIG. 6 , only partsdifferent from FIG. 6 will be described.

Acquisition of an image (step S100), detection of a face (step S102) anddisplay of the image (step S104) are similar to those in FIG. 6 . Afterthe display of the image (step S104), an indicator and the like aredisplayed at step S106. The indicator and the like include the indicatorI1 and the displays L1 and L2 of the numerical values of the indicatoras shown in FIG. 7 .

Next, if the number of detections is not 0 at the determination of thenumber of detected faces at step S110, then the flow proceeds to stepS114, where it is determined whether or not the standardized faceevaluation value of a detection result 1 is equal to or above anumerical value set in advance as a lower limit for display (here, X).Since a frame is to be displayed if the standardized face evaluationvalue of the detection result 1 is equal to or above X, the frame isdisplayed (steps S120 and S186). However, this is the same as in FIG. 6, and description thereof will be omitted. Then, the flow proceeds tostep S190. On the other hand, if the standardized face evaluation valueof the detection result 1 is below X, then a frame is not displayed, andthe flow proceeds to step S190.

The subsequent steps (steps S190, S192, S200, S202 and S210) are similarto those in FIG. 6 .

Thus, an operator can know the face evaluation value of the frameindicating a face in more detail by referring to the indicators. Bysetting the face evaluation value (display determination value) fordisplaying a frame, it is possible to avoid that a frame is displayedfor a target with a low evaluation value which includes noise. Byselecting the display determination value from among values set inadvance or setting it, the determination value for display can bechanged according to purposes. For example, in the case where a face issurely to be captured, such as a case of photographing a face inclose-up, the display determination value can be set high.

Furthermore, by displaying a frame even when the face evaluation valueof a subject is lower in comparison with the face evaluation value of ausually displayed subject, it is possible to avoid sudden change in theframe display and certainly confirm the reliability of display from theindicator.

Second Variation Example of the First Embodiment of Frame Display

FIG. 9 shows an example of the CPU 20 generating a frame displayindicating a face, from the standardized face evaluation value and theposition of a frame indicating a face, outputted from the face detectionsection 30, displaying it on the display section 22, and, whendisplaying an indicator and the like indicating correspondence betweenthe frame display and the standardized face evaluation value, showing adisplay indicating the face evaluation value on the indicator.

As the indicator and the like, an indicator I1 displayed in gradation isshown at the right end, and a display of 50% (L1) and a display of 100%(L2) are shown at the lower right and the upper right, respectively,similar to FIG. 7 . Furthermore, an arrow is displayed to point at theface evaluation value of the frame indicating a face. An operator (or auser) can accurately know the value of the standardized face evaluationvalue by looking at the indicator and the like.

FIG. 10 is a flowchart showing a procedure for the CPU 20 to display aframe continuously or by stages according to the face evaluation valueand display the face evaluation value on an indicator. Since thisflowchart is a variation example of FIG. 8 , only parts different fromFIG. 8 will be described.

Acquisition of an image (step S100), detection of a face (step S102),display of the image (step S104), display of an indicator and the like(step S106), determination of the number of detected faces (step S110),determination on whether or not to perform photographing and processingperformed in the case where photographing is to be performed (steps S200and S202) and end determination processing (step S210) are similar tothose in FIG. 8 . Determination on whether or not to display astandardized face evaluation value of a detection result 1 (step S114)and the display of a frame line in the case where the face evaluationvalue is equal to or above a display determination value (steps S120 andS186) are also similar.

After the frame is displayed at step S186, the standardized faceevaluation value of the detection result 1 is displayed on the indicatorat step S188. The subsequent steps (steps S190 and S192) are similar tothose in FIG. 8 .

Thereby, the operator can know the face evaluation value of the frameindicating a face in more detail from the indicator. Therefore, itbecomes easier to set a display determination value for displaying aframe. Furthermore, by displaying such a portion with a low faceevaluation value as has not been conventionally displayed, as describedabove, it is possible to avoid blinking of the display of the frameindicating a face, and, by displaying the face evaluation value of theframe indicating a face on an indicator, it is possible to show thereliability of the displayed frame.

Third Variation Example of the First Embodiment of Frame Display

FIGS. 11A and 11B show an example of the CPU 20 generating a framedisplay indicating a face, from the standardized face evaluation valueand the position of a frame indicating a face, outputted from the facedetection section 30, displaying it on the display section 22, and, whendisplaying an indicator and the like indicating correspondence betweenthe frame display and the standardized face evaluation value, showing adisplay indicating the face evaluation value on the indicator, withdisplay colors changed according to detection results.

As the indicator and the like, an indicator displayed in gradation anddisplays of numerical values are shown at the right end, and pointers(arrows) pointing at the face evaluation value of the frame indicating aface is shown, similar to FIG. 9 . Furthermore, in order to prevent theindicator and the like from being misread when multiple faces aredetected, the same color is used for the indicator and the like and theframe indicating a face, for each detection result, and different colorsare used for the display of different faces.

In FIG. 11A, a frame A4 indicating the face of a person on the left, anindicator IS and an arrow 16 are displayed in the same color, and aframe A5 indicating the face of a person on the right, an indicator 13and an arrow 14 are displayed in the same color. The color of thenumerical number displays L1 and L2 may be the same as that of any ofthe frames. If a color indicating the detection result 1 is set as thecolor, the display color is favorably prevented from frequentlychanging. The color of the numerical number displays L1 and L2 may be acolor different from the colors of the frames. If a color used todisplay other information about the camera (for example, white) is usedas the color, it is advantageously possible to share circuits orprograms, though it is not shown here.

In FIG. 11B, an indicator 17 is used in common, and the frame A4indicating the face of the person on the left and an arrow 18 aredisplayed in the same color, and the frame A5 indicating the face of theperson on the right and an arrow 19 are displayed in the same color, inorder to make it easy to distinguish. The colors of the numerical valuedisplays L1 and L2 and the indicator 17 may be the same color as any ofthe frames. If a color indicating the detection result 1 is set as thecolor, the display color is favorably prevented from frequentlychanging. The color of the numerical number displays L1 and L2 may be acolor different from the colors of the frames. If a color used todisplay other information about the camera (for example, white) is usedas the color, it is advantageously possible to share circuits orprograms, though it is not shown here.

Thus, an operator can know the standardized face evaluation value ofeach detected face in detail from the indicator.

Fourth Variation Example of the First Embodiment of Frame Display

FIGS. 12A, 12B and 12C show an example of the CPU 20 generating a framedisplay indicating a face from the standardized face evaluation valueand the position of a frame indicating a face, outputted from the facedetection section 30, and displaying it on the display section 22, withthe hue of the frame changed according to the face evaluation value.

Here, the standardized face evaluation value is determined so that it isclassified into any of the three stages of high (FIG. 12A), medium (FIG.12B) and low (FIG. 12C). In the case of “high”, a green frame A6 (FIG.12A) is displayed. In the case of “medium” and “low”, a yellow frame A7(FIG. 12B) and a red frame A8 (FIG. 12C) are displayed, respectively.Thus, it is possible to know an approximate value of the standardizedface evaluation value from the color of the frame line. Though the frameis displayed in the three stages here, it is possible to set no stage.In such a case, the hue gradually changes.

FIG. 13 is a flowchart of the CPU 20 generating a frame displayindicating a face from the standardized face evaluation value and theposition of a frame indicating a face, outputted from the face detectionsection 30, and displaying it on the display section 22, with the hue ofthe frame changed according to the face evaluation value. Since thisflowchart is a variation example of FIG. 6 , only parts different fromFIG. 6 will be described.

Acquisition of an image (step S100), detection of a face (step S102),display of the image (step S104), determination of the number ofdetected faces (step S110), determination on whether or not to performphotographing and processing performed in the case where photographingis to be performed (steps S200 and S202), deletion of a displayeddetection result and subtraction processing of the number of detections(steps S190 and S192) and end determination processing (step S210) aresimilar to those in FIG. 6 .

If the number of detections is not 0 at the determination of the numberof detected faces at step S110, then it is determined at step S130whether or not the standardized face evaluation value x of the detectionresult 1 is “high” (x>80%). If the face evaluation value x is determinedto be “high”, then the flow proceeds to step S136, where green is set asthe frame color for the detection result. If it is determined that theface evaluation value x is not “high”, then the flow proceeds to stepS132, where it is determined whether or not the face evaluation value xis “medium” (x>60%). Here, if it is determined that the face evaluationvalue is “medium”, then the flow proceeds to step S138, where yellow isset as the frame color for the detection result. If it is determinedthat the face evaluation value x is not “medium”, then the flow proceedsto step S134, where it is determined whether or not the face evaluationvalue x is “low” (x>50%). Here, if it is determined that the faceevaluation value is “low”, then the flow proceeds to step S140, wherered is set as the frame color for the detection result. If it isdetermined that the face evaluation value x is not “low”, that is, ifthe face evaluation value is too low to determine that the target is aface, a frame is not displayed, and the flow proceeds to step S190. Inother cases, the frame indicating the face of the detection result 1 isdisplayed with the frame color set at step S136, S138 or S140. Afterthat, similar processing is repeated the number of times correspondingto the number of detected faces.

Thus, an approximate value of the standardized face evaluation value canbe known from the hue of the frame indicating a face. Furthermore, bydisplaying the frame indicating a face even in the case of a subjectwith a lower face evaluation value than a subject for which the framehas been displayed conventionally, it is possible to avoid blinking ofthe display of the frame. The reliability of the face evaluation valueof a displayed face can be determined from the color.

Fifth Variation Example of the First Embodiment of Frame Display

FIG. 14 shows an example of the CPU 20 generating a frame displayindicating a face from the standardized face evaluation value and theposition of a frame indicating a face, outputted from the face detectionsection 30, and displaying it on the display section 22, with the linesegment display rate of the line segments constituting the frame line ofa frame changed according to the face evaluation value.

Here, the standardized face evaluation value is indicated as the linesegment display rate of the line segments constituting the frame line.For example, when the whole frame line is displayed along the linesegment directions of the frame line, that is, when the frame line isdisplayed by a solid line, the face evaluation value is 100%. When thedisplayed part of the frame line is 50% relative to the unit length,that is, when the frame line is displayed by a dashed line, and the rateof the line segments is 50%, the face evaluation value is 50%. Asanother display method, the rate of portions displayed as a frame linemay be 50% when the whole frame line length is assumed as 100%. Thus, itis possible to instinctively know an approximate value of thestandardized face evaluation value from the display of the frame line.

FIG. 15 is a flowchart of the CPU 20 generating a frame displayindicating a face from the standardized face evaluation value and theposition of a frame indicating a face, outputted from the face detectionsection 30, and displaying it on the display section 22, with the linesegment display rate of the line segments constituting the frame line ofa frame changed according to the face evaluation value. Since thisflowchart is a variation example of FIG. 8 , only parts different fromFIG. 8 will be described.

Acquisition of an image (step S100), detection of a face (step S102),display of the image (step S104), determination of the number ofdetected faces (step S110), determination on whether or not to performphotographing and processing performed in the case where photographingis to be performed (steps S200 and S202), deletion of a displayeddetection result and subtraction processing of the number of detections(steps S190 and S192) and end determination processing (step S210) aresimilar to those in FIG. 8 . Though the step of display of an indicatorand the like (step S106) shown in FIG. 8 is not shown in FIG. 15 , itmay be provided. As for the line segment display rate of the frame line,a value (a rate) indicating it is easy to detect. It is also effectiveto further indicate it by an indicator or the like.

If the number of detections is not 0 at the determination of the numberof detected faces at step S110, then the flow proceeds to step S114,where it is determined whether or not the standardized face evaluationvalue of a detection result 1 is equal to or above a numerical value(here, X) set in advance as a lower limit for display. If thestandardized face evaluation value of the detection result 1 is equal toor above X, a frame is to be displayed, and therefore, the flow proceedsto step S142. Here, a frame is displayed with a frame line having a linesegment display rate according to the standardized face evaluation valueof the detection result 1. After that, the flow proceeds to step S190.On the other hand, if the standardized face evaluation value of thedetection result 1 is below X, the frame is not displayed, and the flowproceeds to step S190. Since the steps after step 190 are the same asthose in FIG. 8 , description thereof will be omitted.

Thus, an approximate value of the standardized face evaluation value canbe known from the line segment display rate of the frame line of theframe indicating a face. Furthermore, by displaying the frame indicatinga face even in the case of a subject with a lower face evaluation valuethan such a subject for which the frame has been conventionallydisplayed, it is possible to avoid blinking caused due to display andnon-display of a frame. In addition to that it is possible to determinethe reliability of the face evaluation value of a displayed face fromthe line segment display rate of the frame line, it is also possible toavoid annoying display of the frame even when the frame is continuouslydisplayed because the displayed part is decreased in the case of a lowface evaluation value.

Sixth Variation Example of the First Embodiment of Frame Display

FIG. 16 shows an example of the CPU 20 generating a frame displayindicating a face from the standardized face evaluation value and theposition of a frame indicating a face, outputted from the face detectionsection 30, and displaying it on the display section 22, with the widthof the line segments constituting the frame line of the frame changedaccording to the face evaluation value.

Here, the standardized face evaluation value is indicated as the widthof a frame line. When the face evaluation value is high, the frame lineis displayed thick, and when the face evaluation value is low, the frameline is displayed thin. As the method for thickening the frame line,there are a method of determining the position to display the frame lineand then increasing the width with the position as the center, a methodof increasing the width only inward from the frame line and a method ofincreasing the width only outward from the frame line. In general, themethod of increasing the width with the position of the frame line asthe center is often a function provided for the CPU 20 in advance, andit is advantageous in that the program is simplified. The method ofincreasing the width only inward from the frame line is advantageous inthat, when there are multiple faces positioned close to one another,interference with the display of the other frames can be avoided. Themethod of increasing the width only outward from the frame line isadvantageous in that confirmation of the face displayed within the frameis not interfered with.

When the face evaluation value is indicated by the width of a frameline, it is possible to determine whether or not the face evaluationvalue is large or small from an approximate width, that is, from whetherthe width is thick or thin. However, if it is desired to display theface evaluation value more accurately, it is recommended to display anindicator. An indicator I10 is displayed at the right end of FIG. 16 inwhich the top end indicates the upper limit of the line width of a frameline to be displayed and the bottom end indicates the lower limit of theline width. Values for associating the line width and the faceevaluation value may be displayed on the indicator. Here, there aredisplayed a value L2 indicating the upper limit of the line width at thetop end and a value L1 indicating the lower limit of the line width atthe bottom end. Of course, intermediate values may be displayed, or ascale may be provided on the indicator I10. The face evaluation valuepointed at by an arrow may be displayed on the indicator I10.

Thus, it is possible to instinctively know an approximate value of thestandardized face evaluation value from the display of the width of aframe line. It is not necessarily the whole line that is to be displayedwith a changed width. For example, a part where the width of a linesegment is changed and a part where the width is not changed mayalternately appear at predetermined intervals. It is possible to preventthe frame line from being thin and difficult to visually confirm.

FIG. 17 is a flowchart of the CPU 20 generating a frame displayindicating a face from the standardized face evaluation value and theposition of a frame indicating a face, outputted from the face detectionsection 30, and displaying it on the display section 22, with the frameline width of the frame changed according to the face evaluation value.Since this flowchart is a variation example of FIG. 8 , only partsdifferent from FIG. 8 will be described.

Acquisition of an image (step S100), detection of a face (step S102),display of the image (step S104), display of an indicator and the like(step S106), determination of the number of detected faces (step S110),determination on whether or not to perform photographing and processingperformed in the case where photographing is to be performed (steps S200and S202), deletion of a displayed detection result and subtractionprocessing of the number of detections (steps S190 and S192) and enddetermination processing (step S210) are similar to those in FIG. 8 . Itdoes not matter whether or not the step of display of an indicator andthe like (step S106) exists.

If the number of detections is not 0 at the determination of the numberof detected faces at step S110, then the flow proceeds to step S114,where it is determined whether or not the standardized face evaluationvalue of a detection result 1 is equal to or above a numerical value(here, X) set in advance as a lower limit for display. If thestandardized face evaluation value of the detection result 1 is equal toor above X, a frame is to be displayed, and therefore, the flow proceedsto step S144. Here, a frame is displayed with a frame line having a linewidth according to the standardized face evaluation value of thedetection result 1. For example, a line with a 10-dot width is displayedin the case of a face evaluation value of 100%, and a line with a 5-dotwidth is displayed in the case of a face evaluation value of 50%. Afterthat, the flow proceeds to step S190. On the other hand, if thestandardized face evaluation value of the detection result 1 is below X,a frame is not displayed, and the flow proceeds to step S190. Since thesteps after step S190 are the same as those in FIG. 8 , descriptionthereof will be omitted.

Thus, an approximate value of the standardized face evaluation value canbe known from the frame line width of the frame indicating a face.Furthermore, it is possible to continuously display the frame, avoidingblinking caused due to the conventional operation of switching betweendisplay and non-display of a frame. In addition to that it is possibleto determine the face evaluation value of a displayed face from theframe line width, it is also possible to know it in more detail if anindicator is displayed.

Because a displayed part decreases in the case of a low face evaluationvalue, it is possible to avoid annoying display of the frame even whenthe frame is continuously displayed.

Seventh Variation Example of the First Embodiment of Frame Display

FIGS. 18A, 18B and 18C show an example of the CPU 20 generating a framedisplay indicating a face from the standardized face evaluation valueand the position of a frame indicating a face, outputted from the facedetection section 30, and displaying it on the display section 22, withthe temporal display rate of the frame line changed according to theface evaluation value. Here, the standardized face evaluation value isindicated as the temporal display rate of a frame line. If the faceevaluation value is 67%, the frame is displayed so that the time forwhich the frame is displayed is 67% (D1 and 18A) relative to a certainunit time and the time for which the frame is not displayed is 33% (D2and 18B). If the unit time is short, a new blinking problem occurs.Therefore, it is desirable that the unit time is long to some extent.Naturally, the image is continuously displayed (D3).

FIG. 19 is a flowchart of the CPU 20 generating a frame displayindicating a face from the standardized face evaluation value and theposition of a frame indicating a face, outputted from the face detectionsection 30, and displaying it on the display section 22, with thetemporal display rate of the frame line of the frame changed accordingto the face evaluation value. Since this flowchart is a variationexample of FIG. 8 , only parts different from FIG. 8 will be described.

Acquisition of an image (step S100), detection of a face (step S102),display of the image (step S104), determination of the number ofdetected faces (step S110), determination on whether or not to performphotographing and processing performed in the case where photographingis to be performed (steps S200 and S202), deletion of a displayeddetection result and subtraction processing of the number of detections(steps S190 and S192) and end determination processing (step S210) aresimilar to those in FIG. 8 . Though the step of display of an indicatorand the like (step S106) shown in FIG. 8 is not shown in FIG. 19 , itmay be provided.

If the number of detections is not 0 at the determination of the numberof detected faces at step S110, then the flow proceeds to step S114,where it is determined whether or not the standardized face evaluationvalue of a detection result 1 is equal to or above a numerical value setin advance as a lower limit for display (here, X). If the standardizedface evaluation value of the detection result 1 is equal to or above X,a frame is to be displayed, and the flow proceeds to step S146. Here,the frame for the detection result 1 is blinkingly displayed at a timeinterval obtained by multiplying a reference time set in advance by thestandardized face evaluation value of the detection result 1. Forexample, if the standardized face evaluation value is 100%, the frame iscontinuously displayed. If the standardized face evaluation value is50%, the frame is displayed for 0.5 seconds and is not displayed for theremaining 0.5 seconds, in the case where the reference time is 1 second.Here, the description is made for step S146. However, actually, the stepS146 means that a routine is continuously running.

After that, the flow proceeds to step S190. On the other hand, if thestandardized face evaluation value of the detection result 1 is below X,the frame is not displayed, and the flow proceeds to step S190. Sincethe steps after step S190 are the same as those in FIG. 8 , descriptionthereof will be omitted.

Thus, the conventional display method using two kinds of displaystatuses of display and non-display of the frame is not performed, butthe frame is continuously blinking, and an approximate value of thestandardized face evaluation value can be known from the rate of timefor which the frame is displayed.

Eighth Variation Example of the First Embodiment of Frame Display

FIG. 20 shows an example of the CPU 20 generating a frame displayindicating a face from the standardized face evaluation value and theposition of a frame indicating a face, outputted from the face detectionsection 30, and displaying it on the display section 22, with thestandardized face evaluation value displayed as a numerical value (L3)together with the frame line. Here, the standardized face evaluationvalue is displayed with 100% as a reference. However, the referencevalue for standardization can be selected according to necessities. Forexample, the standardized face evaluation value may be indicated with 10as a reference. It may not be indicated by %. Furthermore, thestandardized face evaluation value may be combined with any other frameline display or may be combined with an indicator. It is advantageousthat the standardized face evaluation value can be detected morespecifically.

FIG. 21 is a flowchart of the CPU 20 generating a frame displayindicating a face from the standardized face evaluation value and theposition of a frame indicating a face, outputted from the face detectionsection 30, and displaying it on the display section 22, with thestandardized face evaluation value displayed as a numerical value (L3)together with the frame line. Since this flowchart is a variationexample of FIG. 8 , only parts different from FIG. 8 will be described.

Acquisition of an image (step S100), detection of a face (step S102),display of the image (step S104), determination of the number ofdetected faces (step S110), determination on whether or not to performphotographing and processing performed in the case where photographingis to be performed (steps S200 and S202), deletion of a displayeddetection result and subtraction processing of the number of detections(steps S190 and S192) and end determination processing (step S210) aresimilar to those in FIG. 8 . Though the step of display of an indicatorand the like (step S106) shown in FIG. 8 is not shown in FIG. 21 , itmay be provided.

If the number of detections is not 0 at the determination of the numberof detected faces at step S110, then the flow proceeds to step S114,where it is determined whether or not the standardized face evaluationvalue of a detection result 1 is equal to or above a numerical value(here, X) set in advance as a lower limit for display. If thestandardized face evaluation value of the detection result 1 is equal toor above X, a frame is to be displayed, and therefore, the flow proceedsto step S186. Here, a frame indicating a face is displayed, and afterthat, the standardized face evaluation value of the detection result 1is displayed at step S187. Though the standardized face evaluation valueis displayed as a numerical value here, it may be displayed in adifferent manner. For example, it is possible to indicate thestandardized face evaluation value by a shape such as a pie chart or bythe size of an indicator in a particular shape.

After that, the flow proceeds to step S190. On the other hand, if thestandardized face evaluation value of the detection result 1 is below X,a frame is not displayed, and the flow proceeds to step S190. Since thesteps after step S190 are the same as those in FIG. 8 , descriptionthereof will be omitted.

Thus, by displaying the frame indicating a face if a face evaluationvalue can be calculated and it is equal to or above a certain value, itis possible to avoid the conventional display method using two kinds ofdisplay statuses of display and non-display of the frame but performcontinuous display of the frame. Furthermore, by displaying theevaluation value, a standardized face evaluation value can be known.

These variation examples can be embodied in combination with oneanother. By combining multiple examples, smoother display can berealized. For example, it is possible to change the frame line widthwhile changing the frame line shade intensity. Of course, thecombination is not limited to the combination of these two examples, andcombination of more than two examples is also possible. At step S186,the color or the shade intensity of the frame may be changed accordingto the standardized face evaluation value, and it is also possible to,when a frame is to be displayed, display a constant frame irrespectiveof the face evaluation value.

Next, description will be made on an example in which the display of theframe indicating a face is caused to change with the lapse of time ifthe standardized face evaluation value changes from a value equal to orabove the display determination value to a value below the displaydetermination value or if the standardized face evaluation value changesfrom a value below the display determination value to a value equal toor above the display determination value, at the time of displaying theframe indicating a face on the display section 22 of the image-takingapparatus 1.

Second Embodiment of Frame Display

FIG. 22 shows an example of the CPU 20 generating a frame displayindicating a face from the standardized face evaluation value and theposition of a frame indicating a face, outputted from the face detectionsection 30, and causing it to change with the lapse of time whendisplaying it on the display section 22. In this example, thestandardized face evaluation value is indicated by the shade intensityof the frame line. That is, when the standardized face evaluation valueis small, the frame line is displayed with a low shade intensity, andwhen the standardized face evaluation value is large, the frame line isdisplayed with a high shade intensity.

For example, when the frame line display determination value is X %, andthe face evaluation value changes from a value equal to or above thedisplay determination value to a value below the display determinationvalue, the display of the frame is not stopped instantaneously, but theframe line gradually disappears. In FIG. 22 , the figure at the left endindicates that the standardized face evaluation value exceeds thedisplay determination value, and in this case, the frame line indicatingthe face is displayed, for example, with a shade intensity based on theevaluation value (A20). In this case, if, when the standardized faceevaluation value equal to or above the display determination value andthe frame indicating a face is displayed, the face evaluation valuechanges and becomes below the display determination value, and the frameline indicating a face is hidden instantaneously, then the frame line isdisplayed again when the standardized face evaluation value exceeds thedisplay determination value again, and the display blinks. Therefore, bycausing the display to disappear with the lapse of time so that ittransitions from A21 to A24, it is possible to avoid sudden change inthe display of the frame indicating a face. When the frame line displaydetermination value is X %, and the face evaluation value changes from avalue below the display determination value to a value equal to or abovethe display determination value, the frame is not displayedinstantaneously, but the frame line gradually appears. To makedescription with reference to FIG. 22 , the figure at the right endindicates that the standardized face evaluation value is below thedisplay determination value, and the frame indicating a face is notdisplayed in the figure (A24). In this case, if, when the standardizedface evaluation value exceeds the display determination value, the frameline indicating a face is displayed instantaneously, the frame line ishidden again when the standardized face evaluation value is below thedisplay determination value again, and the display blinks. Therefore, bycausing the display to appear with the lapse of time so that ittransitions from A24 to A20, it is possible to avoid sudden change inthe display of the frame indicating a face. That is, even if the faceevaluation value changes from a value equal to or above the displaydetermination value to a value below the display determination value,the frame indicating a face does not suddenly disappear but graduallydisappears. Such display makes it possible not only to avoid suddendisplay change but also to make only slight display change even when theface evaluation value transitions around the display determinationvalue, and thereby smooth display can be realized.

Thus, by causing the frame to appear or disappear continuously or bystages with the lapse of time when change from non-display to display ofthe frame is determined or when change from display to non-display isdetermined, it is possible to smoothly display the frame, avoiding theconventional sudden display change between display and non-display ofthe frame.

Next, description will be made on an example in which the display of theframe indicating a face is caused to change with the lapse of time ifthe standardized face evaluation value changes from a value equal to orabove the display determination value to a value below the displaydetermination value or if the standardized face evaluation value changesfrom a value below the display determination value to a value equal toor above the display determination value, at the time of displaying theframe indicating a face. The change from non-display to display with thelapse of time and the change from display to non-display with the lapseof time are the same in the content though they are different in thechange direction. Therefore, description will be made representativelyon the processing performed when the standardized face evaluation valuechanges from a value equal to or above the display determination valueto a value below the display determination value here. Though a case maybe assumed where multiple faces are detected, description will be madeon the assumption that only one face has been detected to avoidcomplicatedness of the description. In the case of multiple faces,processing is to be performed according to the face evaluation value ofeach of the detected faces. For example, a frame indicating one face iskept being displayed because its face evaluation value is equal to orabove the display determination value, and a frame indicating anotherface gradually disappears because its face evaluation value is below thedisplay determination value. Such an operation will be understood byapplying the description below to the display of each face.

FIG. 23 is a flowchart of the CPU 20 of causing the display of the frameindicating a face to change with the lapse of time if the standardizedface evaluation value changes from a value equal to or above the displaydetermination value to a value below the display determination value orif the standardized face evaluation value changes from a value below thedisplay determination value to a value equal to or above the displaydetermination value, at the time of displaying the frame indicating aface.

First, 0 is substituted for variables A and B to be used, forinitialization (step S90). Then, signal processing is performed for animage signal obtained by inputting an image from or taking an image withthe image input section 10, at an input signal processing section toacquire the image signal as an image (step S100). Next, at the facedetection section, particular targets of a subject included in theinputted image, that is, a face, eyes, pupils, a nose, a mouth and thelike are detected from the inputted image, and the feature points andpatterns of them are identified to detect the face of the subject (stepS102). Next, the image inputted from the image input section 10 isdisplayed (step S104). Next, at step S112, it is determined whether aface has been detected or not. Here, for the determination on whether aface has been detected or not, the face evaluation value may be used, oran evaluation value other than the face evaluation value may be used. Ifit is determined at step S112 that a face has been detected, then theflow proceeds to step S114, and otherwise, the flow proceeds to stepS200.

At step S114, it is determined whether or not the standardized faceevaluation value of the detected face is equal to or above a numericalvalue (here, X) set in advance as a lower limit for display. If thestandardized face evaluation value is equal to or above X, a frame is tobe displayed, and therefore, the flow proceeds to step S120. Here, thestandardized face evaluation value of the detected face is substitutedfor the frame line shade intensity of the detection result, and a frameindicating the face is displayed at step S185. Then, at step S189, thestandardized face evaluation value of the detected face and itsdetection position are substituted for the variables A and B,respectively, and the flow proceeds to step S200.

At step S114, if the standardized face evaluation value of the detectedface is below the numerical value (here, X) set in advance as a lowerlimit for display, a frame is not to be displayed, and therefore, theflow proceeds to step S300.

At step S300, it is determined whether or not the detection position ofthe detected face is close to the variable B. If the detection positionis close to the variable B, then it is determined that the detected faceis the same face detected last, and the flow proceeds to step S320. Ifthe detection position is different from the variable B, there is apossibility that a different face has been detected, and therefore, theprocessing is not performed, and the flow proceeds to step S500. Here,the processing of causing the frame indicating a face to disappear withthe lapse of time is not performed when there is a possibility that adifferent face has been detected. However, there is also a possibilitythat the subject person has suddenly moved, and therefore, it isconceivable to omit this step S300 and perform the processing.Furthermore, though the determination is made at the position where aface has been detected, the determination may be made based on whetherthe face is the face detected immediately before the detection if eachface can be detected. That is, it is possible that the processing forcausing the frame to disappear with the lapse of time is performed ifthe face is the same face detected immediately before the detection, andthe processing is not performed if the face is different from the facedetected immediately before the detection.

At step S320, “variable A-variable Y” is set as the shade intensity ofthe frame line for the detected face. Here, the frame line shadeintensity is indicated between 0 and 255, and the detected faceevaluation value is standardized by 255. Since the value of Y is a valuewhich decreases the value of A with the lapse of time, it is generally avalue set by the system. However, it may be set by an operator. If aface is detected immediately before the detection, a standardized faceevaluation value has been substituted for A. Therefore, by calculatingA-Y, the frame line shade intensity of a new frame line is decreased.

Next, at step S330, a frame for the detection result is displayed. Inthis case, the value set at step S320 is used as the frame line shadeintensity.

After that, at step S340, “A-Y” is newly substituted for the variable A,and at step S500, the face detection position is substituted for thevariable B. Then, the flow proceeds to step S200.

At step S200, whether or not to perform photographing is determined.Commonly, the determination is made based on whether the shutter buttonor the recording button (either of them is not shown) provided on theoperation section 26 has been pressed or not. When the photographingprocessing is to be performed, the photographing processing is performedat step S202, and then, the flow proceeds to step S210. When thephotographing processing is not to be performed, the flow proceeds tostep S210, and it is determined whether or not to end the processing. Ifthe processing is not to be ended, then the flow returns to step S100and repeats similar processing.

Consequently, it is possible to cause the display of the frameindicating a face to change with the lapse of time if the standardizedface evaluation value changes from a value equal to or above the displaydetermination value to a value below the display determination value orif the standardized face evaluation value changes from a value below thedisplay determination value to a value equal to or above the displaydetermination value, at the time of displaying the frame indicating aface, and it is possible to avoid blinking of the display and smoothlydisplay the frame indicating a face.

Here, acquisition of an image (step S100), detection of a face (stepS102) and display of the image (step S104) are performed in timesequence for convenience of description with reference to the flowchart.However, it is also possible to perform display of the image (step S104)while performing detection of a face (step S102), for example, in thecase of such a configuration that the face detection section operatesindependently from the CPU 20, and it is rather desirable in order toshorten the entire processing. Not only for the parts described abovebut also for the parts described below, higher speed can be realized byperforming parallel processing if possible.

Here, an image is displayed on the display section 22 first, and then aframe is displayed on the display section 22. However, it is alsopossible to first synthesize an image and a frame to be displayed andthen send the synthesized result to the display section 22 to display itthereon, depending on the configuration of the apparatus.

Furthermore, it is not necessarily required to display an image. Forexample, in the case of an image-taking apparatus for monitoringintended to monitor whether the face of a person exists or not, it issufficient to display only a face frame, without displaying an image.Because it is sufficient to display the image only when it is determinednecessary to do so from the displayed frame. For example, the followingis conceivable. When a face is detected, a frame is displayed. Then, ifthe detection continues for a predetermined period of time (for example,three seconds or more), then an image is displayed for the first time.This makes it possible to monitor not a monitoring picture screen onwhich a lot of targets are displayed and which requires alertness butonly a necessary monitoring screen.

Here, when it is determined that the frame indicating a face is to bedisplayed, a frame with a frame line shade intensity corresponding tothe standardized face evaluation value is displayed, and when changefrom display to non-display occurs, the frame is displayed with theframe line shade intensity changed, with the lapse of time, from theframe line shade intensity corresponding to the standardized faceevaluation value of the last display. Therefore, smooth display whichdoes not give an uncomfortable feeling can be realized. However, this isnot limiting. When the frame indicating a face is displayed, it may bedisplayed as in any of the embodiment described here.

First Variation Example of the Second Embodiment of Frame Display

FIG. 24 shows the display shown when the standardized face evaluationvalue changes from a value equal to or above the display determinationvalue to a value below the display determination value or when thestandardized face evaluation value changes from a value below thedisplay determination value to a value equal to or above the displaydetermination value, at the time of displaying the frame indicating aface on the display section 22 of the image-taking apparatus 1. If theface evaluation value becomes below the display determination value, theframe line of the frame indicating a face is displayed, with the linesegment display rate of the line segments constituting the frame linechanged with the lapse of time (from A30 to A34).

FIG. 25 is a flowchart of the CPU 20 generating a frame displayindicating a face from the standardized face evaluation value and theposition of a frame indicating a face, outputted from the face detectionsection 30, displaying it on the display section 22, and, if thestandardized face evaluation value changes from a value equal to orabove the display determination value to a value below the displaydetermination value or if the standardized face evaluation value changesfrom a value below the display determination value to a value equal toor above the display determination value, causing the line segmentdisplay rate of the line segments constituting the frame line of theframe indicating a face to change with the lapse of time. Since thisflowchart is a variation example of FIG. 23 , only parts different fromFIG. 23 will be described.

Acquisition of an image (step S100), detection of a face (step S102),display of the image (step S104), determination on detection of a face(step S112), determination on whether or not a standardized faceevaluation value is equal to or above a display determination value(step S114), display of a frame for the detection result (step S185),update of variables (step S189), determination on whether or not toperform photographing and processing performed in the case wherephotographing is to be performed (steps S200 and S202) and enddetermination processing (step S210) are similar to those in FIG. 23 .

At initialization of variables (step S92), 0 is substituted for avariable B and a variable n to initialize them.

If it is determined at step S114 that a frame is not to be displayed,determination of the detection position is made (step S300). If thedetection position is close to the variable B, and it is determined thatthe face is the same face, then the flow proceeds to step S350. If thedetection position is significantly different from the variable B, thenthe flow proceeds to step S500.

At step S350, 1 is added to the value of the variable n. After that, atstep S360, the imaginary-part rate of dashed line is calculated. Theimaginary-part rate of dashed line is calculated by K=1−Z×n. Here, n isan integer, and Z is a constant equal to or below 1. However, Z may beset by an operator. Because n is added at step S350, the imaginary-partrate of dashed line decreases every time the processing through S350 toS370 is repeated, and the display rate relative to the unit length ofthe frame line decreases. After that, at step S370, the frame for thedetection result is displayed by a line with the imaginary-part rate ofdashed line K, and at step S500, the detection position is substitutedfor B. Then, the flow proceeds to step S200.

As described above, it is possible to display the frame indicating aface with the imaginary-part rate of dashed line of the frame line tochanged with the lapse of time if the standardized face evaluation valuechanges from a value equal to or above the display determination valueto a value below the display determination value or if the standardizedface evaluation value changes from a value below the displaydetermination value to a value equal to or above the displaydetermination value, at the time of displaying the frame indicating aface, and it is possible to avoid blinking of the display and smoothlydisplay the frame indicating a face.

Though description is made on the assumption that, when it is determinedthat the frame indicating a face is to be displayed, a normal frame isdisplayed, the frame to be displayed may change according to the faceevaluation value. For example, in the case of this embodiment, byindicating the imaginary-part rate of dashed line by the standardizedface evaluation value when the frame indicating a face is to bedisplayed, and changing the numerical value of the imaginary-part rateof dashed line from the imaginary-part rate of dashed line of the lastdisplay when change from display to non-display occurs, smooth displaywhich does not give an uncomfortable feeling can be realized.

Second Variation Example of the Second Embodiment of Frame Display

FIG. 26 shows the display shown when the standardized face evaluationvalue changes from a value equal to or above the display determinationvalue to a value below the display determination value or when thestandardized face evaluation value changes from a value below thedisplay determination value to a value equal to or above the displaydetermination value, at the time of displaying the frame indicating aface on the display section 22 of the image-taking apparatus 1. If theface evaluation value becomes below the display determination value, theframe line of the frame indicating a face is displayed, with the frameline width changed with the lapse of time (from A40 to A44).

FIG. 27 is a flowchart of the CPU 20 generating a frame displayindicating a face from the standardized face evaluation value and theposition of a frame indicating a face, outputted from the face detectionsection 30, displaying it on the display section 22, and, if thestandardized face evaluation value changes from a value equal to orabove the display determination value to a value below the displaydetermination value changes from a value below the display determinationvalue to a value equal to or above the display determination value,causing the frame line width of the frame indicating a face to changewith the lapse of time. Since this flowchart is a variation example ofFIG. 25 , only parts different from FIG. 25 will be described.

Initialization of variables (step S92), acquisition of an image (stepS100), detection of a face (step S102), display of the image (stepS104), determination on detection of a face (step S112), determinationon whether or not a standardized face evaluation value is equal to orabove a display determination value (step S114), display of a frame forthe detection result (step S185), update of variables (step S189),determination on whether or not to perform photographing and processingperformed in the case where photographing is to be performed (steps S200and S202) and end determination processing (step S210) are similar tothose in FIG. 25 .

Determination of the detection position (step S300), increment of thevariable n (step S350) in the case where the detection position isdetermined to be close to the variable B, and substitution of thedetection position for the variable B (step S500) performed in the casewhere the detection position is not determined to be close and afterdisplay of the frame, which is to be described later, are similar tothose in FIG. 25 .

After the increment of the variable n (step S350), a line width W iscalculated at step S362. The line width is calculated by W=Y×(1−X×n).Here, Y is a reference line width, n is an integer, and X is a constantequal to or below 1. Y may be expressed as a maximum line width. Y and Xmay be set by an operator. Because n is added at step S350, the linewidth W decreases every time the processing through S350 to S370 isrepeated, and the frame line gradually thins down. After that, at stepS370, the frame for the detection result is displayed with the linewidth W, and at step 500, the detection position is substituted for B.Then, the flow proceeds to step S200.

As described above, it is possible to cause the frame line width of theframe indicating a face to change with the lapse of time if thestandardized face evaluation value changes from a value equal to orabove the display determination value to a value below the displaydetermination value or if the standardized face evaluation value changesfrom a value below the display determination value to a value equal toor above the display determination value, at the time of displaying theframe indicating a face, and it is possible to avoid blinking of thedisplay and smoothly display the frame indicating a face.

Third Variation Example of the Second Embodiment of Frame Display

FIG. 28 shows an example of displaying the frame indicating a face, withthe display time rate of the frame indicating a face changed with thelapse of time (from D4 to D8) if the standardized face evaluation valuechanges from a value equal to or above the display determination valueto a value below the display determination value or if the standardizedface evaluation value changes from a value below the displaydetermination value to a value equal to or above the displaydetermination value, at the time of displaying the frame indicating aface on the display section 22 of the image-taking apparatus 1.

FIG. 29 is a flowchart of the CPU 20 generating a frame displayindicating a face from the standardized face evaluation value and theposition of a frame indicating a face, outputted from the face detectionsection 30, displaying it on the display section 22, and, if thestandardized face evaluation value changes from a value equal to orabove the display determination value to a value below the displaydetermination value or if the standardized face evaluation value changesfrom a value below the display determination value to a value equal toor above the display determination value, causing the display time rateof the frame line of the frame indicating a face to change with thelapse of time. Since this flowchart is a variation example of FIG. 25 ,only parts different from FIG. 25 will be described.

Initialization of variables (step S92), acquisition of an image (stepS100), detection of a face (step S102), display of the image (stepS104), determination on detection of a face (step S112), determinationon whether or not a standardized face evaluation value is equal to orabove a display determination value (step S114), display of a frame forthe detection result (step S185), update of variables (step S189),determination on whether or not to perform photographing and processingperformed in the case where photographing is to be performed (steps S200and S202) and end determination processing (step S210) are similar tothose in FIG. 25 .

Determination of the detection position (step S300), increment of thevariable n (step S350) in the case where the detection position isdetermined to be close to the variable B, and substitution of thedetection position for the variable B (step S500) performed in the casewhere the detection position is not determined to be close and afterdisplay of the frame, which is to be described later, are similar tothose in FIG. 25 .

After the increment of the variable n (step S350), a display time rate Tis calculated at step S362. The display time rate T is calculated byT=1−U×n. Here, n is an integer, and U is a constant equal to or below 1.However, the constant U may be set by an operator. Because n is added atstep S350, the display time rate decreases every time the processingthrough S350 to S374 is repeated, and the display time graduallydecreases. After that, at step S374, the frame for the detection resultis displayed with the display time rate T, and at step S500, thedetection position is substituted for B. Then, the flow proceeds to stepS200.

As described above, it is possible to cause the display time rate of theframe indicating a face to change if the standardized face evaluationvalue changes from a value equal to or above the display determinationvalue to a value below the display determination value or if thestandardized face evaluation value changes from a value below thedisplay determination value to a value equal to or above the displaydetermination value, at the time of displaying the frame indicating aface, and it is possible to avoid blinking of the display and smoothlydisplay the frame indicating a face.

Fourth Variation Example of the Second Embodiment of Frame Display

FIG. 30 shows an example of displaying the frame indicating a face, withthe in-screen display rate of multiple line segments constituting theframe indicating a face changed with the lapse of time (from A50 to A54)if the standardized face evaluation value changes from a value equal toor above the display determination value to a value below the displaydetermination value or if the standardized face evaluation value changesfrom a value below the display determination value to a value equal toor above the display determination value, at the time of displaying theframe indicating a face on the display section 22 of the image-takingapparatus 1. Furthermore, if the standardized face evaluation valuechanges to a value below the display determination value or a valueequal to or above the display determination value, the in-screen displayrate is changed by changing the distance of the positions of themultiple line segments of the frame indicating a face from the center ofthe frame according to the lapse of time from the point of time when thestandardized face evaluation value changes.

As shown in FIG. 30 , the frame indicating a face is displayed when theface evaluation value is equal to or above the display determinationvalue (A50). If the face evaluation value decreases and becomes equal toor below the display determination value, the frame is separated intomultiple line segments constituting the frame indicating a face, andthen each line segment moves outward from the center of the frame,rotating at random, as shown in A51.

Due to such display, when change from display to non-display occurs, theframe indicating a face does not suddenly disappear but graduallydisappears from the screen, and smooth display change can be realized.

It is not necessarily required that each line segment rotates. Withoutthe rotation, the amount of calculation performed for display by the CPU20 can be reduced advantageously. However, if each component is rotated,an impression of the frame disappearing explosively can be given. Inorder to give the impression of more explosive deletion, it is desirableto set the rotation direction and speed of each line segment at randomand rotate each line segment in the same direction and at the same speedduring the change of the display. The speed at which each line segmentmoves outward from the center of the frame indicating a face may beconstant. The speed may be initially high and then a little lower, or itmay be initially low and then a little higher. In the case of displayingthe line segments moving at a constant speed, a more explosiveimpression can be given. In the case of displaying the line segmentsinitially at a high speed and then at a lower speed, it is easy toexpress the line segments gathering from around the face when the faceevaluation value decreases, becomes below the display determinationvalue and then increases again. In the case of displaying the linesegments initially at a low speed and then at a higher speed, it ispossible to reduce change in the display of the frame indicating a facewhen the face evaluation value fluctuates around the displaydetermination value, and smooth display can be realized.

Here, an example is shown in which multiple line segments of the frameindicating a face move outward from the center of the frame indicating aface. However, the respective line segments may move toward the centerof the frame indicating a face. In this case, since there may be a faceas a subject at the center, it is desirable that decrease of the shadeintensity of the frame line and the like occurs as the respective linesegments move toward the center.

FIG. 31 is a flowchart of the CPU 20 generating a frame displayindicating a face from the standardized face evaluation value and theposition of a frame indicating a face, outputted from the face detectionsection 30, displaying it on the display section 22, and, if thestandardized face evaluation value changes from a value equal to orabove the display determination value to a value below the displaydetermination value or if the standardized face evaluation value changesfrom a value below the display determination value to a value equal toor above the display determination value, causing the in-screen displayrate of each of the line segments constituting the frame line of theframe indicating a face to change with the lapse of time. Since thisflowchart is a variation example of FIG. 25 , only parts different fromFIG. 25 will be described.

First, 0 is substituted for the variable B to be used, forinitialization (step S92). Next, the rotation direction and the rotationspeed of each line segment are set (step S96). In this embodiment, sincethe settings are made each time, the change in the frame is differenteach time, and the change in the display of the frame with the lapse oftime is different each time. Therefore, a variety of displays can beprovided. However, the display may be determined in advance. In thiscase, advantageously, the amount of calculation is reduced, and a smoothoperation can be realized. Acquisition of an image (step S100),detection of a face (step S102), display of the image (step S104),determination on detection of a face (step S112), determination onwhether or not a standardized face evaluation value is equal to or abovea display determination value (step S114), display of a frame for thedetection result (step S185), update of variables (step S189),determination on whether or not to perform photographing and processingperformed in the case where photographing is to be performed (steps S200and S202) and end determination processing (step S210) are similar tothose in FIG. 25 .

At step S300, determination of the detection position is made. If it isdetermined that the detection position is close to the variable B, thenthe position and the amount of rotation of each line segment arecalculated at step S366. Here, the calculation is performed so that anin-screen display rate set in advance changes. However, there may be acase where, when display is switched, the in-screen display rate doesnot change even though the display temporarily changes (such as thechange from A50 to A51 in FIG. 30 ). After that, at step S376, fragmentsof the frame are displayed. The substitution of the detection positionfor the variable B (step S500) after that is similar to FIG. 25 .

As described above, it is possible to cause the in-screen display rateof the frame line of the frame indicating a face to change with thelapse of time if the standardized face evaluation value changes from avalue equal to or above the display determination value to a value belowthe display determination value or if the standardized face evaluationvalue changes from a value below the display determination value to avalue equal to or above the display determination value, at the time ofdisplaying the frame indicating a face, and it is possible to avoidblinking of the display and smoothly display the frame indicating aface.

The frame line becomes fragments, and they gradually disappear from thescreen. Thereby, effective display can be realized.

Furthermore, these variation examples can be embodied in combinationwith one another. FIG. 32 is an example of causing the in-screen displayrate of the frame line of the frame indicating a face to change with thelapse of time and also causing the frame line shade intensity to changewith the lapse of time (from A60 to A64) if the standardized faceevaluation value changes from a value equal to or above the displaydetermination value to a value below the display determination value orif the standardized face evaluation value changes from a value below thedisplay determination value to a value equal to or above the displaydetermination value, at the time of displaying the frame indicating aface. By combining two kinds of changes, smoother display change can berealized.

In addition, it is also possible to combine displaying the frameindicating a face continuously or by stages according to the faceevaluation value and causing display to change continuously or by stageswith the lapse of time if the face evaluation value becomes below thedisplay determination value. For example, it is possible to display theframe indicating a face with a width equal to or above a predeterminedline width when the face evaluation value is equal to or above thedisplay determination value and cause the line width to decrease fromthe predetermined line width with the lapse of time when the faceevaluation value becomes below the display determination value. Thereby,smoother display of the frame can be realized. If multiple displaychanges can be combined, for example, if the frame is displayed with theline width changed when the face evaluation value is equal to or abovethe display determination value and displayed with the frame line shadeintensity and the line width changed when the face evaluation value isbelow the display determination value, smoother and more effectivedisplay can be realized.

Though the frame indicating a face has been described as a continuousline or a set of line segments surrounding a face in the above examples,there are other kinds of frames. FIGS. 33A, 33B, 33C, 33D, 33E, 33F 33Gand 33H show an example of the frame and the change in the frame. FIG.33A shows an example where the frame is constituted by four startssurrounding a face. In this case, it is the size that corresponds to theline width, and a frame A71 in FIG. 33A changes to a frame A72 in FIG.33B.

FIG. 33C shows an example of a frame which is indicated only by theupper left corner and the lower right corner of a face. Here, an exampleis shown where the frame is displayed with the frame line shadeintensity changed from that of a frame A73 in FIG. 33C to that of aframe A74 in FIG. 33D.

FIGS. 33E and 33F show an example where hemispherical frames A75 and A76are displayed at opposite corners of a face. FIG. 33E shows an examplewhere the line segment display rate of the frame A75 is 100%, and FIG.33F is an example where the line segment display rate is 20%. Becausethe frame is a hemisphere, the line segment direction is at right anglesto the radius, and such display is shown. Here, an example is shownwhere the hemisphere disappears gradually starting from its center linepart as the line segment display rate decreases. Reversely, thehemisphere may disappear starting from its periphery parts.

FIG. 33G shows an example where the face evaluation value is indicatedby a circle A77. When the face evaluation value changes, it is indicatedby change in the size as the change from the circle A77 in FIG. 33G to acircle A78 in FIG. 33H. Though the change is indicated by the size, itmay be indicated by the shade intensity.

Here, description has been made on the assumption that the particulartarget is a face. However, the present invention is applicable to anyparticular target of a subject, for example, a part of a person, such asa head, eyes, pupils, red eyes, closed eyes, glasses, brows, a mouth andlips, or the whole person. Furthermore, the present invention isapplicable to a lot of various targets, for example, a car, an animaland a plant such as a flower. As an example, the present invention canbe used for observation of plants and may be used for an apparatus forimage-taking and display of flowering and the like. Furthermore, thepresent invention is applicable not only to an image-taking apparatusbut also to a monitoring apparatus as described above. Furthermore, apart of display may not be shown by a display section but expressed withthe use of voice by a speaker section.

The invention claimed is:
 1. An image display apparatus comprising: animage-taking element that converts an optical image into an imagesignal; a display that displays a display image based on the imagesignal; and one or more processors configured to detect a face includedin the image signal according to a face evaluation value indicating afeature of the face; generate frame display information including faceevaluation value information indicating whether a frame line is to bedisplayed surrounding the detected face; control the display to displaya frame line around the detected face based on the generated framedisplay information; identify a same detected face corresponding to asame person as a previously detected face, the previously detected facebeing a face detected before a current face detecting process, the faceevaluation value of the previously detected face being still availableto the one or more processors; control the display to erase, for thedetected face identified as the same detected face, the frame line witha lapse of time; identify a new detected face that is different from thepreviously detected face; control the display to display a new frameline around the new detected face without erasing the new frame linewith a lapse of time; control the image-taking element to focus on thedetected face; and repeat the face detecting process.
 2. The imagedisplay apparatus according to claim 1, wherein the frame displayinformation generated by the one or more processors causes a frame lineto change according to a lapse of time from a point of time when adetermination to switch from display to non-display is made or a pointof time when a determination to switch from non-display to display ismade.
 3. The image display apparatus according to claim 1, wherein theone or more processors are further configured to control the display todisplay the new frame line corresponding to the face evaluation value ofthe new detected face.
 4. An image display method comprising: convertingan optical image into an image signal; detecting a face included in theimage signal according to a face evaluation value indicating a featureof the face; generating frame display information including a faceevaluation value information indicating whether a frame line is to bedisplayed surrounding the detected face; displaying a frame line aroundthe detected face based on a comparison of the generated frame displayinformation with a threshold; identifying a same detected facecorresponding to a same person as a previously detected face, thepreviously detected face being a face detected before a current facedetecting process, the face evaluation value of the previously detectedface being still available; erasing the frame line, for the detectedface identified as the same detected face, with a lapse of time;identifying a new detected face that is different from the previouslydetected face; displaying a new frame line around the new detected facewithout erasing the new frame line with a lapse of time; controlling animage-taking element to focus on the detected face; and repeating theface detecting process.
 5. The image display method according to claim4, wherein the generated frame display information causes a frame lineto change according to a lapse of time from a point of time when adetermination to switch from display to non-display is made or a pointof time when a determination to switch from non-display to display ismade.
 6. The image display method according to claim 4, furthercomprising displaying the new frame line corresponding to the faceevaluation value of the new detected face.